Field of the Invention
The present invention is related to a technique for, based on images that are captured from a plurality of viewpoint positions, generating an image from a set virtual-viewpoint position.
Description of the Related Art
A technique of obtaining a three-dimensional model of an object (for example, a person or the like) from images captured by a plurality of cameras, and generating, from a plurality of captured images, an image (hereinafter referred to as a virtual-viewpoint image) at an arbitrary viewpoint (hereinafter referred to as a virtual-viewpoint) at which no camera is actually present is known (for example, Japanese Patent No. 5011224).
At this time, the shape of the object in the virtual-viewpoint image is obtained from the three-dimensional model, and a region that its shape encompasses is generated by texture mapping. The color of a pixel used in this texture mapping is obtained by referencing a real image (hereinafter referred to as a reference image) that is close to the virtual-viewpoint in the line of sight direction with a high weighting.
While the position and line of sight direction, and camera parameters, such as the focal length, of the plurality of cameras are known, a certain amount of error is included in these. Accordingly, a three-dimensional model of an object obtained from the plurality of cameras naturally also includes error. As a result of this, a pixel in the reference image referenced when deciding the color of a pixel positioned in an object in the virtual-viewpoint image, and in particular in the outline (edge portion) of such an object, is not necessarily positioned within that object. For example, a pixel corresponding to a reference image when obtaining a color of a pixel of the edge portion of the face of a person in the virtual-viewpoint image may end up being a pixel outside of the face of that person (for example, a pixel representing a floor). When such a situation occurs, an object (and in particular, its outline) in a virtual-viewpoint image that is generated ends up being an unnatural color, and image degradation cannot be avoided.